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Abstract. In this paper we prove symmetry results for classical 
solutions of nonlinear cooperative elliptic systems in a ball or in 
annulus in M N , N > 2. More precisely we prove that solutions 
having Morse index j < N — 1 are foliated Schwarz symmetric if the 
nonlinearity has a convex derivative and a full coupling condition 
' is satisfied along the solution. 



1. Introduction and statement of the results 

We consider a semilinear elliptic system of the type 



1.1^ 



-AU = F(\x\,U) in ft 
U = ondQ 



where F = (/i,...,/ m ) is a function belonging to C 1,a ([0,+oo) x 
IR m ;IR m ), Q is a bounded domain in and U = (ui, . . . ,u m ) is a 
vector valued function in Q, m, N > 2. 

Systems of this type arise in many applications in different fields (see 
e.g. m- TO- 

In this paper we continue the study of the symmetry of solutions of 
(11. ip in rotationally symmetric domains, namely when Q is a ball or 
an annulus, started in [7|. 

We recall that if Q is a ball and the system is cooperative then in 
[H], [H], [9], [23] it is proved that every positive solution U (i.e. m > 
for any i = 1 , . . . m) is radial and radially decreasing if every fi is 
non increasing with respect to This result is obtained through 
the famous "moving plane method" ([21]), as in the scalar case Qllj). 
However this method does not allow neither to consider sign changing 
solutions nor nonlinearities which are not radially nonincreasing with 
respect to \x\. Moreover, as it is a continuation method, it does not 
provide symmetry results in the annulus. 

Another symmetry result for cooperative systems in a ball, obtained 
by the symmetrization method, is proved in [T3] . 
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In the scalar different approach was introduced in [T7j and 

later extended in [TS] and [T2] (see also [IB]) which allows to cover 
cases which cannot be treated by using the moving plane method. This 
approach is essentially based on Morse index considerations and on 
some convexity properties of the nonlinearity. The symmetry obtained 
for the solutions of fll.lD with this method is an axially symmetry which 
is indeed what one expects. 

It is then natural to try to extend the method of [T7] to the case of 
systems like (11. ip . However this extension presents several difficulties, 
as we explain below, and cannot be done straightforwardly. In [7] 
we obtained symmetry results in this direction for solutions of (II. ip 
imposing some assumptions on the nonlinearity which in particular 
imply that it is convex in all variables. For some kind of systems (e.g. 
power type odd nonlinearities) this hypothesis does not allow to get 
symmetry of sign changing solutions. 

Here we will consider other type of nonlinearities, as in [12J, which 
cover this case. Moreover in [Z], if /// > .'5 the nonlinearity should 
be the sum of functions depending only on two variables. For the 
nonlinearities we consider here we do not need this assumption. 

To present our results and to be more precise we need some prelim- 
inary definitions. 

DEFINITION 1.1. Let Q be a rotationally symmetric domain in 
Mr, N > 2. We say that a continuous vector valued function U = 
(ui, . . . ,u m ) : Q — > M m is foliated Schwarz symmetric if each compo- 
nent Ui is foliated Schwarz symmetric with respect to the same vector 
p G M N . In other words there exists a vector p G M. N , \p\ = 1, such 

that U(x) depends only on r = \x\ and 9 = arccos ^j^j • pj and U is 

(componentwise) nonincreasing in 9. 

Remark 1.1. Let us observe that if U is a solution of (II. ip and the 
system satisfies some coupling conditions, as required in Theorem 11.21 
then the foliated Schwarz symmetry of U implies that either U is ra- 
dial or it is strictly decreasing in the angular variable 9. This will be 
deduced by the proof of Theorem 11.21 

DEFINITION 1.2. Let U be a C 2 (fi; R m ) solution of f lTTf . 

i) We say that U is linearized stable (or that it has zero Morse 
index) if the quadratic form 



for any * = (fa, . . . ,i/j m ) G C c 1 (0;M m ) where J F (x,U) is the 
jacobian matrix of F computed at U. 




(1.2) 
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. . . ,u m ) > V (x,ui, . . .,u m ) G Q' x 



ii) U has (linearized) Morse index equal to the integer \i = fi(U) > 
1 if /J, is the maximal dimension of a subspace of C*(f2;R m ) 
where the quadratic form is negative definite. 

iii) U has infinite (linearized) Morse index if for any integer k there 
is a A;- dimensional subspace of C*(f2;IR m ) where the quadratic 
form is negative definite. 

DEFINITION 1.3. • We say that the system (JET) is coopera- 
tive or weakly coupled in an open set Q' C Q if 

for any i,j = 1, . . . , m with i ^ j. 
• We say that the system (11 .11) is fully coupled along a solution U 
in an open set Q' C Q if it is cooperative in fi' and in addition 
V/, J C {1, ... , m} such that 1^0, J ^ 0, I n J = 0, I U J = 
{1, . . . , m} there exist z G /, jo G J such that 

meas ({x G fi' : ^(Id, Z7(a;)) > 0}) > 
du jo 

Note that the previous definition means that the linearized system 
at a solution U is weakly or fully coupled. 

Let e G S' Ar ~ 1 be a direction, i.e. e G M. N , \e\ = 1, and let us define 
the set 

Q(e) = {x G Q : x ■ e > 0} 



In [7] we proved the following result 

THEOREM 1.1 ( [7J ). Let Q be a ball or an annulus m R N , N > 
2, and let U G C 3 > a (Q;R m ) be a solution of ( II. ip with Morse index 
lx(U) < N . Moreover assume that: 

i) The system is fully coupled along U in fl(e), for any e G S 1 ^^ 1 . 

ii) For any i,j — 1, . . .m ^(\x\,Ux, . . . ,u m ) is nondecreasing in 
each variable u^, k = 1, . . . , m, for any \x\ G Vt. 

iii) if m > 3 then, for any i G {1, . . . , m}, fi(\x\,ui, . . . , u m ) = 
^2 kj ti9ik(\x\,Ui,u k ) where g ik G C 1,Q ([0, +oo) x 1R 2 ). 

Then U is foliated Schwarz symmetric and if U is not radial then it is 
strictly decreasing in the angular variable (see Definition \l.l\) . 

The monotonicity hypothesis ii) implies that each fi is convex with 
respect to each variable Uj, i,j = 1, . . . , m. 

Note that the regularity of the solution U in Theorem ll.l[ as in the 
next one, is a consequence of the regularity of the nonlinearity F which 
is assumed to be of class C 1,a . Moreover it implies that the derivatives 
of U are classical solutions of the linearized system which is used in the 
proof. 
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Though the results of Theorem 11.11 also applies to sign changing 
solutions of f ll.ip there are cases when the nonlinearity F, if considered 
on the whole M m is not convex in the Uk variables. This is, for example, 
the case of 

F(ui,u 2 ) = (fi(ui,u 2 ), f 2 (ui,u 2 )) = (\u 2 \ p ~ 2 u 2 , , p,q>l 

As announced we consider here other nonlinearities which cover this 
case and we also do not assume iii) of Theorem 11.11 when m > 3. More 
precisely our symmetry results are the following. 

THEOREM 1.2. Let tt be a ball or an annulus in R^, N >2, and let 

U G C 3 ' a (J}; R m ) be a solution of (JTTp with Morse index fi(U) < 2V-1. 
Moreover assume that: 

i) The system is fully coupled along U in Q 

ii) For any i,j = l,...m the function J^(|a;|, S) is convex in S = 
(si , . . . , s m ) . 

g( W ,«- + (i- tW <«g( W ,ff) + (i- t )g(W,^ 

for any t G [0, 1], S' , S" G R m and x G Q. 
Then U is foliated Schwarz symmetric and if U is not radial then it is 
strictly decreasing in the angular variable (see Definition 



Remark 1.2. As observed before the assumption ii) of Theorem 11.21 
allows to get the symmetry of solutions in cases not covered by Theorem 
II. H and also to remove the assumption iii) for m > 3. 
On the contrary the assumption on the Morse index is more restrictive 
since we require (jl(U) < N — 1. Note that in the scalar case, in [Tj5] 
(see also [T2] , [IB] ) we could get the symmetry result up to Morse index 
less than or equal to N. For systems some serious difficulty arises and 
we are not able to consider the case (jl(U) = N. 



THEOREM 1.3. Under the assumptions of Theorem ] assume that 
U is a nonradial solution of ( II. ip and either 

a) U has Morse index one 

or 

b) there exist i ,j G {1, . . . , m} such that the function ^j^MM, S) 
satisfies the following strict convexity assumption: 

(1.3) ^(\x\,tS' + (1 - t)S") < t^(\x\,S') + (1 - t)^(\x\, S") 
du jo du jo du jo 

for any t G (0,1), whenever x G f2 and S f , S" G lR m satisfy 
s' k 7^ s'l for any k G {1, . . . , m}. 
Then necessarily 



= 1 
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for any i = 1, . . . , m, whith (r, 9) as in Definition \l.l\ 
In particular if m = 2 then fll.4p implies that 

(1.5) ^(\x\,U(x)) = ^(\x\,U(x)) , Vxefi. 

OU2 OUi 

Note that (11.4)) and (II. 5p were also deduced in [7J under the assump- 
tions of Theorem 11.11 but only for Morse index one solutions. 

Remark 1.3 (Radial symmetry of stable solutions). The symmetry 
result of Theorem 11.21 holds in particular for stable solutions of (11. ip . 

However in this case it is easy to get that the solution is radial 
without any assumption on the nonlinearity (see [7J, Theorem 1.2). 

The general strategy to get symmetry results for systems, using the 
ideas of [FT] and [TP] for the scalar case, is described in [7J. Let us just 
point out here that one of the main difficulties encountered is that the 
linearized operator Ly (see Section 3 for the definition) is not in general 
selfadjoint, due to the fact that the Jacobian matrix Jp(|x|, U) is not 
symmetric. To bypass this difficulty, as in [7] we associate to Jf(\x\, U) 
its symmetric part \ (Jf{\x\, U) + J F (\x\, U)) where J F is the transpose 
of the matrix J F . To this symmetric part is associated a selfadjoint 
operator Lu whose spectrum can be variationally characterized. The 
crucial, simple remark in [7] is that the quadratic form associated to 
the linearized operator Ljj is the same as the quadratic form associated 
to Lu. 

Obviously if the jacobian matrix J F (\x\,U) is symmetric the two linear 
operators coincide. This happens, for example, when the system is of 
gradient type, i.e. when F = Vg for some scalar function g (see [9]), 
since, in this case, the linearized operator corresponds to the second 
derivative of a suitable associated functional. 

However this is not the case for many interesting systems, like e.g. 
the so called hamiltonian systems (see [9]). 

We refer to [7] for further comments on this issue and on the analysis 
of the Morse index of solutions of (11.11) . 

The outline of the paper is the following. In Section 2 we recall some 
definitions and results about linear systems. In Section 3 we prove 
some preliminary results for solutions of (II. ip and prove Theorem 11.21 
and Theorem 11.31 Finally in Section 4 we present a few examples. 

2. Preliminaries on linear systems 

In this section we recall several known results, almost all are proved 
in the paper [7J. 

Let Q be any smooth bounded domain in M. N , N > 2, and D a m x m 
matrix with bounded entries: 

(2.1) D = (dtf)S=i , dy e L°°(fi) 
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Let us consider the linear elliptic system 

in fi 
on dfi 



(2.2) 



-AU + D(x)U = F 
U = 



i.e. 



-Ait! + d n ui H h d lm u m = fi 



in fi 



"mm ">m 



U, 



in fi 
on dfi 



-Au m + d mX u x H h d„ 

^ M X = • • • = U m = 

where F = (A, . . . , f m ) G (L 2 (fi)) m , U = (u u . . . , u m ). 

This kind of linear system appears in the linearization of the semilinear 

elliptic system (II. II) . 

DEFINITION 2.1. The system (J22D is said to be 

• cooperative or weakly coupled in if 

(2.3) djj < Oa.e. in fi, whenever i ^ j 

• fully coupled in fi if it is weakly coupled in fi and the following 
condition holds: 

(2.4) 

V/, J C {l,...,m},J, J^0, In J = 0, JU J= {l,...,m} 

3z € / , jo G : meas ({x G fi : dj j < 0}) > 

It is well known that either condition (12. 3p or conditions (12.31) and 
(12. 4p together are needed in the proofs of maximum principles for sys- 
tems (see [8], [10], [22] and the references therein). In particular if both 
are fulfilled the strong maximum principle holds as it is shown in the 
next theorem. 

Notation remark: here and in the sequel inequalities involving 
vectors should be understood to hold componentwise, e.g. if \I/ = 
(■01, . . . , if) m ), ^ nonnegative means that ipj > for any index j = 
l,...,m. 

THEOREM 2.1. (Strong Maximum Principle and Hopf's Lemma). 
Suppose that (IO) . (1231) and fl23D fcoZd and U = (ui, . . . , w m ) G C^fi; M m ) 
a u>ea£; solution of the inequality 

-AU + D(x)U > in fi 

ie. 
(2.5) 



> 



i=l *j=i 

/or any nonnegative ^ = (ipi, . . . , ip m ) G C*(fi; M m ). 

If U > in Q, then either U = in Q or U > in Q. In the latter 
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case if P G <9f2 and U(P) = then f^-(-P) < ; where v is the unit 
exterior normal vector at P. 

We are interested in the quadratic form associated to system (12.21) . 
namely 

Q(tf ; f2)= I [\V^\ 2 + D{x){^^)]dx = 
Jn 

(2.6) 

|V^i| 2 + Y dijix^i^j dx 

_i=l *J=1 

for * G C^R" 1 ) ( or * G ^(^R™) ). 

It is easy to see that this quadratic form coincides with the quadratic 
form associated to the symmetric system 

-AU + C(x)U = F infi 
U = on 90 



(2.7) 



i.e. 



-Aui + cnui H h c lm u m = fx 

Au m -\- C m \U\ -\- • • • -\- C mm U m fn 



where 



1 



(2.8) C = -(D + D t ) i.e. C = ( Cij ), Cij = - (d tj + d Jt ) 

So to study the sign of the quadratic form Q we can also use the prop- 
erties of the symmetric system (12. 7p . 

Therefore we review briefly the spectral theory for this kind of sim- 
metric systems, and use it to prove some results that we need for the 
possible nonsymmetric system ( 12. 2p . 

Remark 2.1. If system ( 12.21) is cooperative, respectively fully coupled, 
so is the associate symmetric system (12. 7p . 

Let O be a bounded domain in Mr, N > 2, and consider for m > 1 
the Hilbert spaces L 2 = L 2 (fi) = (L 2 (Q)) m , = Hj(f2) = (H^Q)) m , 
where if / = (fi,...,f m ), g = (g u ...,g m ) the scalar products are 
defined by 



(2.9) 



m m „ 

Lf,g)v = ^2{fi,9i)i^(a) = Y / fididx 
i=i i=i Jn 

m m „ 

(/>»)hJ = £(/«,9i)i(J(!i) = Y / V ^ ' V ^ rfx 
i=i *=i 



Let C = C(x) = (cjj(x))™ = i a symmetric matrix whose elements are 
bounded functions: 

(2.10) c^ G L°° , Cjj = Cji a.e. in Q 
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and consider the bilinear form 
(2.11) 



B(U,$)= [ [W • V$ + C(U, $)] = / 
Jn Jn 



Cij U{ (pj 



_i=l *J = 1 

As recalled and discussed in [7], by the spectral theory of self adjoint 
operators there exist a sequence {Xj} of eigenvalues, with — oo < Ai < 
A 2 < • • • , limj_j. +00 Xj = +oo, and a corresponding sequence of eigen- 
functions {W j }, W j G Hq Pi C 1 ^;^" 1 ) that weakly solve the systems 



(2.12) 



-AW j + CW j = XjW j in ft 
W j = ondil 



i.e. if = (wx, . . . , w n 

-Awi + c u wi + h ci m w m = AjWi 



-Aw m + c ml wi H h c mm u> m = Ajto 



that satisfy the following properties. In what follows if Q' is a subdo- 
main of fl we denote by Afe(O') the eigenvalues of the same system with 
Q substituted by Q'. 

PROPOSITION 2.1. Suppose toat C = (c^)S=i sa*is/iea fl2~T0|) . 
and Zet {Aj}, {VF- 7 } 6e i/ie sequences of eigenvalues and eigenfunctions 
that satisfy (1232]) . 
Define the Rayleigh quotient 

(2.13) W = ^ /or-VeH* V^O 

with B(., .) as in ( 12.1 ip . Then the following properties hold, where Vk 
denotes a k- dimensional subspace of Hq and the orthogonality condi- 
tions V-LWk or V-L Vk stand for the orthogonality in L 2 . 

i) Ai = mm V eHl,v^o R ( v ) = mm veHl ,(vy) L 2=i B ( v > v ) 

ii) A m = rnin V( z H i v ^ V±Wly v±Wm _ 1 R(V) 

= mm veHl,(v,v) L2 =i,v±w 1 ,...,v±w m - 1 B ( V i V ) l f m > 2 

iii) X m = min Vm max VeVm )Vj lqR(V) 

iv) A m = max Vm _ 1 mm v ±ev m -i ,v^oR(V) 

v) IfWe Hi, W ^ 0, and R(W) = Ai, then W is an eigenfunc- 
tion corresponding to X\. 

vi) lim meas ( n/)^o Ai(fi') = +oo 

vii) If the system is cooperative in Q and W is a first eigenf unction, 
then W + and W~ are eigenfunctions, if they do not vanish. 

viii) If the system is fully coupled in Q, then the first eigenfunction 
does not change sign in Q and the first eigenvalue is simple, 
i.e. up to scalar multiplication there is only one eigenfunction 
corresponding to the first eigenvalue. 
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ix) Assume that the system is fully coupled in £1, C = (c^) 

is another matrix that satisfies (I2.10p . and let {A'^}, be the 
sequence of eigenvalues of the corresponding system. Ifcij > d \j 
for any i,j — l,...,m then X 1 > X\. 

Let us turn back to the (possibly) nonsymmetric cooperative system 
( 12. 2 p with the matrix D = (c?jj)™ =1 satisfying 

(2.14) dij G L°°{Q) , dij < , whenever i ^ j 

In the sequel we shall indicate by A^ = A^(— A + D; Q) the eigen- 
values of the associated symmetric system (12.71) . Analogously the cor- 
responding eigenfunctions will be indicated by Wj . 
We also denote the bilinear form associated with the symmetric system 
(IZIDby 

m m 

CijUi<pj 

As already remarked, the quadratic form (12.61) associated to the system 
(12. 2p coincides with that associated to system (12. 7p . i.e. 

Q(m-tt)= ( |W| 2 + Z>(:r)(#,#) = B s ^,q) 
Jn 

if ^ G H^(n;R m ). 

DEFINITION 2.2. We say that the maximum principle holds for the 
operator —A + D in an open set fi' C fi if any U G H 1 (Q / ) such that 

• U < on dW (i.e. U + G Hj(fi')) 

• -AU + D(x)U < in Q' (i.e. / W ■ V$ + D(x)(U, $) < for 
any nonnneg ative $ G Hj(fi')) 

satisfies J7 < a.e. in Q. 

Let us denote by \f\tt') > the sequence of the eigenvalues of the 
symmetric system in an open set Q' C Q. 

THEOREM 2.2. [Sufficient condition for weak maximum principle] 
Under the hypothesis (I2.14p . if \{ s \tt') > then the maximum princi- 
ple holds for — A + D in Q' C Q. 

Almost immediate consequences of the previous theorem are the fol- 
lowing "Classical" and "Small measure" forms of the weak maximum 
principle (see HO], |2D], 

COROLLARY 2.1. i) // (12. 3p holds and D is a.e. nonnegative 
definite in Q' then the maximum principle holds for —A + D in 

n>. 



[W • W + C(U, $)] 
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ii) There exists 5 > 0, depending on D, such that for any sub- 
domain Q' C Q the maximum principle holds for —A + D in 
Q' C Q provided \Q'\ < 8. 

Obviously the converse of Theorem 12.21 holds if D = C is symmetric: 
if the maximum principle holds for —A + C in Q' then A^(fi') > 0. 
In fact if A^O') < since the system is cooperative (and symmetric) 
there exists a nontrivial nonnegative first eigenfunction $i > 0, $ ^ 
0, and the maximum principle does not hold, since — A$ x + C $i = 
Ai$i < in Q', $! = on dQ', while $i > and $i ^ 0. 
However this is not true for general nonsymmetric systems. Roughly 
speaking the reason is that there is an equivalence between the validity 
of the maximum principle for the operator —A + D and the positivity 
of its principal eigenvalue Ai, whose definition is given below, and the 
inequality Ai(fi') > which can be strict, holds. 

More precisely we recall that the principal eigenvalue of the operator 
— A + D in an open set Q' C Q is defined as 

Ai(fi') = sup{A GM:3fG Wf c f (tf; IT) s.t. 
¥>0-Atf + .D(s)*-A#>0infi'} 

(see [B] and the references therein, and also [I] for the case of scalar 

equations). 

We then have: 

PROPOSITION 2.2. Suppose that the system (Q is fully coupled 
in an open set Q' C.Q. Then: 

i) there exists a positive eigenfunction ^ e W^' c (fi'; ]R OT ) which 
satisfies 

(2.16) 

- A*x + £>(x)^i = Ai(n')^i in ft, *i>0mO', = on dQ' 

Moreover the principal eigenvalue is simple, i.e. any function 
that satisfy (12.161) must be a multiple of^\. 

ii) the maximum principle holds for the operator —A + D in Q' if 
and only if Ai(fi') > 

iii) if there exists a positive function ^ e W^(Q'; M m ) such that 
* > 0, -Atf + D(x)V > in Q' , then either Ai(ft') > or 
Ai(fi') = and ^ = c^i /or some constant c. 

iv) Ai(fi') > A( (£]'), wift equality if and only if^i is also the first 
eigenfunction of the symmetric operator —A + C m O' , C — 
^(D + D l ). If this is the case the equality C(x)^\ = D(x)^fi 
holds and, if m = 2, this implies that d 12 = d 2 i- 
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3. Proof of symmetry results 

Let us now consider the system (II. lft : 

-AU = F(\x\,U) in ft 
U = on on 

where ft is a ball or an annulus in M. N , F = . . . , f m ) is a function 
belonging to C 1,a ([0, +oo) x M m ; R m ) and U — (u\, . . . , u m ) is a vector 
valued function in ft, m, iV > 2. 

In Section 1 we defined the Morse index of a solution through the 
quadratic form Qu defined in (II. 2p associated to the linearized operator 
at a solution U, i.e. to the linear operator 

(3.1) Lu(V) = -AV - J F (\x\,U)V 

As remarked, it coincides with the quadratic form corresponding to the 
selfadjoint operator 

(3.2) Lu(V) = -AV - l - {J F (\x\,U) + MHU)) V 

where J F is the transpose of the matrix Jp. 

Hence if \k and W k denote the symmetric eigenvalues and eigen- 
functions of Ljj, i.e. W k satisfy 

-AW k + CW k = X k W k in ft 
W k = on 9ft , 

where C = c i: j(x), Cij(x) = I f|(M, U{x)) + U(x)) , 

as in the scalar case it is easy to prove (see [7]) the following 

PROPOSITION 3.1. Let ft be a bounded domain in R N . Then the 
Morse index of a solution U to ( II .ip equals the number of negative 
symmetric eigenvalues of the linearized operator Lu . 



3.1. Preliminary results. Let e G S N 1 be a direction, i.e. e G M. N , 
|e| = 1, and let us define the hyperplane T(e) and the "cap" ft(e) as 

T(e) = {x G R N : x ■ e = 0} , ft(e) = {x G ft : x ■ e > 0} 

Moreover if x G ft let us denote by a e (x) the reflection of x through 
the hyperplane T(e) and by U Ue the function U o a e . 



LEMMA 3.1. • Assume that U is a solution of (II. ip and that 
the system is fully coupled along U in ft. Let us define for any 
direction e G S N ^ the matrix B e (x) = (b^(x)) . , where 

b%{x) = - [ |£ [\x\,tU(x) + (l-t)U a *(x)) dt 

JO vUj 



(3.3) 
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Then the function W e = U — U ae = (wi, . . . , w m ) satisfies (in 
Q and) in Q(e) the linear system 

-AW + B e (x)W =0 infl(e) 
W = on dn{e) 

which is fully coupled in Q and Q(e) for any e G S ,7V ~ 1 . 

// also hypothesis ii) of Theorem holds, and we define for 
any direction e G S ,7V_1 B e,s (x) = (b^^x)^ , where 

1 fdf in , TT/ » , df, 



(3-4) b*f(x) = -- ^-(\x\,U(x)) + CT-(x)) 

then the linear system with matrix B e,s is fully coupled as well 
in Q and Q(e) for any e G S N ~ 1 . 
Moreover for any i,j = 1, . . . , m and iGll 



(3.5) bUx) > b%(x 



and the inequality is strict for any i , j such that l^ 2 - satisfies 

the strict convexity assumption (\1.3\i if Uk{x) ^ w£ e (x) for any 
k G {1, . . . , m). 

As a consequence for the quadratic forms Q e and Q e,s associated 
to the matrixes B e and B e,s we have that 



(3.6) = Q e (W e ;tt(e))= [ [\ VW e \ 2 + B e (W e , W e )] dx 

Jn(e) 

> [ [\VW e \ 2 + B e ' s (W e ,W e )] dx = Q e ' s {W e -n{e)) 
Jn(e) 

with strict inequality if F satisfies the hypothesis b) of Theorem 
\l.tft and Wj: 7^ for any k G {1, . . . , m}. 

Proof. From the equation —AU = F(\x\,U(x)) we deduce that the 
reflected function U ae satisfies the equation — AU Ue = F(\x\, U a "(x)) 
and hence for the difference W e = U — U Ue = (iii — u° e , . . . , u m — u^) 
we have -AW e = F(\x\,U) - F(\x\, U ae ) so that 

-Aw i = f i (\x\,U)-f i (\x\,U < ") = 



J2 f 7T W x \> tu ( x ) + ( X - t)U ae (x)\ dt( Uj - uf) 

j=1 JO " U 3 

and (13.31) follows. 

Since the system (11. ip is weakly coupled, J^- 1 > for any i ^ j , so 
that b\j < and the system (I3.3P is weakly coupled as well. 
To see that the system (I3.3P is also fully coupled let us show that if 
io,jo £ {1, • • -, m } are such that |^-(|x|, U(x)) > for some x G Q, 
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then bf o j o (x) < 0. This follows immediately by the nonnegativity and 

the continuity of using the definition of 6^ oJo . This implies that 

(13. 3 p is fully coupled in O and, since B is symmetric with respect to 
the reflection a e , it is fully coupled in 0(e) as well. 
Moreover if hypotheses ii) of Theorem 11.21 holds then 

(3.7) -b\ j (x)= [ ^-[\x\,tU(x) + (l-t)U^(x)]dt 
Jo v u j 

- I! (* If [N ^ (a;)] + (i ■ t} ff ^ u "^) dt 



1 f ()fi i\x\M*)) + ^(\*lu° m (*))) = 



2 \duj ' duj 
This implies (I3.5P and the inequality is strict for any i , j such that 



9 fig 
'-'jo 



-7^- satisfies the strict convexity assumption (II. 3p if Uk(x) ^ u^. e (x) for 



any k G {1, . . . , m}. 
This in turn implies the full coupling of the system with matrix B e,s 
and (13.61) if hypothesis b) of Theorem 11.31 holds. □ 

Next, we state the following two lemmas, whose proofs can be found 
in[7]. 

LEMMA 3.2. Let U = (uy, . . . , u m ) be a solution of (11.1 p and assume 
that the hypothesis i) of Theorem holds. If for every e G S N ~ l we 
have either U > U <7e in 0(e) or U < U r7e in 0(e), then U is foliated 
Schwarz symmetric. 

LEMMA 3.3. Let U = (ui, . . . , u m ) be a solution of (II. ip and assume 
that the hypothesis i) of Theorem \1.2\ holds. 

Suppose that there exists a direction e such that U is symmetric with 
respect to T(e) and the principal eigenvalue Ai(0(e)) of the linearized 
operator Ljj(V) = —AV — Jp(x, U)V in 0(e) is nonnegative. Then U 
is foliated Schwarz symmetric. 

LEMMA 3.4. Suppose that U is a solution of (II. ip with Morse index 
/J,(U) < N — 1 and assume that the hypothesis i) of Theorem M ."A holds. 
Let Q e,s be the quadratic form associated to the operator L e,s {V) = 
-AV + B e ' s V B e ' s being defined in flUD ■ 

(3.8) 

Q e ' s (^;0') = / [\v^\ 2 + B e > s (m 1 m)]dx = 
£ [t iv*l 2 - Jj (f§(W,^)) + g(M.0"(*») m 

Then there exists a direction e G S^ -1 such that 

Q e < s (^;0(e)) >0 



dx 
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for any \1/ G Cl(Q(e); W" 1 ). Equivalently the first symmetric eigen- 
value \{(L e ' s ,n(e)) of the operator L e ' s (V) = -AV + B e ' s V in ft(e) 
nonnegative (and hence also the principal eigenvalue Xi(L e,s , Q(e)) is 
nonnegative). 

Proof. Let us assume that 1 < j = fJ.(U) < N — 1 and let $i, . . . , $ 3 
be mutually orthogonal eigenfunctions corresponding to the negative 
symmetric eigenvalues X\(Ljj, Q), . . . , X s ALu, Q) of the linearized oper- 
ator Lu(V) = -AV - J F (x, U)V in Q. . 

For any e G S^ -1 let e ' s be the first positive L? normalized eigen- 
function of the symmetric system associated to the linear operator L e,s 
in fl(e). We observe that 4> e,s is uniquely determined since the corre- 
sponding system is fully coupled in Q(e). Let $ e ' s be the odd extension 
of (p e,s to Q, and let us observe that $ _e ' s = — $ e,s , because 5 e,s is sym- 
metric with respect to the reflection a e . 

The mapping e i— > $ 6,s is a continuous odd function from S N ~ 1 to H , 
therefore the mapping h : S N ~ X — > W defined by 

h(e) = ((^ s ,^) LHn) ,...,(^ s ,^) LHn) ) 

is an odd continuous mapping, and since j < N — 1, by the Borsuk- 
Ulam Theorem it must have a zero. This means that there exists a 
direction e G S 1 ^^ 1 such that $ 6,s is orthogonal to all the eigenfunctions 
This implies that Q c/ ($ e '' s ; Q) > 0, because yu(Z7) = j, 
and since $ es is an odd function, we obtain that < Qu(<& e ' s \Q) = 
Q e ' s ($ e > s , Q) = 2Q e ' s (0 e ' s , fi(e)) = 2X\(L e ' s , fi(e)) " □ 

3.2. Proof of Theorems 11.21 and Theorem 11.31 

Proof of Theorem \1. 6 A By Lemma 13.41 there exists a direction e such 
that the first symmetric eigenvalue X\[L e,s ! , f2(e)) of the operator L e,s (V) = 
— AV+B e ' s V in Q(e) is nonnegative, and hence also the principal eigen- 
value Xi(L e ' s ,Q(e)) is nonnegative. 

By (13.51) and Proposition 12.11 ix), the first symmetric eigenvalue 
Af(-A + B e ,Q(e)) of the operator -A + B e in D,(e), B e being de- 
fined in (13. 4p . is also nonnegative. 

If A^-A + B 6 , fl(e)) > 0, then necessarily the difference W e = U-U ae 
must vanish. In fact, since it satisfies the equation (13. 3p . we get for the 
associated quadratic form 

Q e (W e ;tt(e)) = [ [\VW e \ 2 + B e (W e ,W e )] dx = 
Jn(e) 

so by Proposition [271] W e = 0, and U = U a& . This implies that B e = 
B e,s = Jp{\x\, U), so that we find a direction e satisfying the hypotheses 
of Lemma [3.31 and we get that U is foliated Schwarz symmetric. 
If instead Af(-A + B e ,Q(e)) = 0, then necessarily X\(L e ' s , Q(e)) = 
X\(-A + B e ,Q(e)) = 0. 

Let us now remark for future use in the proof of Theorem 11.31 that if 
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hypothesis b) of Theorem 11.31 holds then necessarily W e = 0, so that 
even in this case we find a direction e such that U is symmetric with 
respect to T(e) and not only the principal eigenvalue \%(Lu, fi(e')) 
of the linearized operator in Q(e') is nonnegative, but also the first 
symmetric eigenvalue \\(Lu , fl(e)) = Xl(Lu, fi(— e)) > 0. 
In fact if W e would not vanish, it should be the first eigenfunction of 
the system f)3.3p . which by Lemma l3~7Ll is fully coupled. So it would be 
strictly positive (or negative) and by Lemma I37TI and the hypothesis b) 
we would get 

(3.9) = Q e (W e ;Q(e)) = [ [\VW e \ 2 + B e (W e , W e )] dx 

> [ [\VW e \ 2 + B e ' s (W e ,W e )] dx = Q e > s {W e -n{e)) 

contradicting the nonnegativity of the first symmetric eigenvalue X\(L e,s , f2(e)) 

of the operator L e > s (V) = —AV + B e ' s V in Q(e). 

Then the only case left is when A^(— A + B e ,Q(e)) = , hypothesis 

b) does not hold, and W e does not vanish, so that it must be the first 

symmetric eigenfunction of the system f)3.3p . which is fully coupled. 

This implies that it does not change sign in fi(e) , and assuming that 

e.g. U > U ae then, by the strong maximum principle we have that 

U > U ae in fi(e). 

We now apply, as in [19], [12] and [TJ, the "rotating plane method", 
which is an adaptation of the moving plane method as developed in 
[5] and obtain a different direction e' such that U is symmetric with 
respect to T(e!) and the principal eigenvalue Ai(L C /, f2(e')) of the lin- 
earized operator in Q(e') is nonnegative. Then by Lemma [3.31 we will 
get that U is foliated Schwarz symmetric. 

More precisely, without loss of generality we suppose that e = (0, 0, . . . , 1) 
and for 9 > we set eo = (sin#, 0, . . . , cos6>), so that eo = e, and 
n g = n(e e ), U 6 = f/ CTe «, W 6 = U - f/ CTe «, Let us define 6 = sup{6> e 
[0,7r) : U > U 6 in VLq}. Then necessarily #o < re, since (U — U°)(x) = 
-(U - U n )(a e ^(x)) for any x G Q (and a e ^(x)) G 
Suppose by contradiction that U ^ U e ° in Qg . Then, by Theorem 12. II 
applied to the difference W e ° = (U - U e °), we get that W e ° > in 
Qg . Taking a compact K C Qg whith small measure and such that 
(componentwise) W 9 ° > (rj, . . . ,rj) for some r] > 0, for 6 close to 9 we 
still have that > (§,...,§) in K, while W e > in Q e \ K by the 
weak maximum principle in domains with small measure. This implies 
that for 9 greater than and close to #o the inequality U > U e in Qg still 
holds, contradicting the definition of 9q. Therefore U = U e ° in Qg . 
Observe that the difference W e satisfies the linear system (13. 3p and does 
not change sign for any 9 G [0,#o)> which implies by Proposition 12.21 
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that it is the principal eigenfunction for the system (13. 3p correspond- 
ing to the eigenvalue Ai = 0. As 9 — )■ 9 the system (13.31) tends to the 
linearized system, in the sense that the coefficients —bij(x) tend to the 
derivatives U(x)). Then by continuity the principal eigenvalue 

\i(Lu, Q(eg )) of the linearized operator Ljj(V) = —AV — J F (x, U)V 
in Q{eg ) is zero and the proof of Theorem 11.21 is complete. □ 

Proof of Theorem \1.3[ To prove Theorem ll.3[ let us recall that, as 
observed above, if hypothesis b) holds then we can find a direction 
e such that U is symmetric with respect to T(e) and not only the 
principal eigenvalue Ai(L[/, f2(e)) of the linearized operator in f2(e) is 
nonnegative, but also the first symmetric eigenvalue A^(L(/, f2(e)) = 
A s 1 (L f/ ,fi(-e)) >0. 

The same happens if U is a Morse index one solution. In fact in this 
case for any direction e G S N ^ at least one amongst A^(L[/, f2(e)) 
and A^(Lf/, f2(— e)) must be nonnegative, otherwise taking the corre- 
sponding first eigenfunctions we would obtain a 2-dimensional sub- 
space of CKQW 71 ) where the quadratic form is negative definite, so 
in the symmetry direction e found above we have that X\(Lu, fi(e)) = 
A s 1 (L l/ ,fi(-e)) >0. 

After a rotation, we may assume that e = e 2 — (0, 1, . . . , 0) so that 
T(e) = {x e R N : x 2 = 0} and we may introduce new (cylinder) 
coordinates (r, 9, y 3 , . . . , yx) defined by the relations x = rfcos^ei + 
sin6>e 2 ] + Yh^V^- 

Then the angular derivative Uq of U with respect to 9, extended by 
zero at the origin if Q is a ball, satisfies the linearized system, i.e. 

(3.10) - AU e - J F (\x\,U)U e = in Q(e 2 ). 

Moreover, by the symmetry of U with respect to the hyperplane T(e2), 
we have that Uq is antisymmetric with respect to T(e 2 ) and therefore 
vanishes on T(e 2 ) and since it vanishes on dQ, it vanishes on dQ(e2) as 
well. 

If Xi(Lu, f2(e 2 )) > then, by the maximum principle, the derivatives 
Ug, must vanish, and hence U is radial. So if Ug ^ necessarily, from 
flTTO]) . X 1 (L u ,Q(e 2 )) = Xl(L u ,n(e 2 )) = and by v) of Proposition 
12.21 Ug is the first eigenfunction of the simmetrized system, as well as 
a solution of (I3.10p (let us observe that by the symmetry of U the 
linearized system (13. IQj) is fully coupled in f2(e 2 )). So we get that 
J F (\x\,U)U g = \(J F (\x\,U) + J F (\x\,U))U e , i.e. (JU} and if m = 2, 
since Ug is positive, we get (II. 5p . □ 

4. Some examples 

A first type of elliptic systems that could be considered are those 
of "gradient type" (see [9]), i.e. systems of the type ( II. ip where 
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fj(\x\,U) = U) for some function g G C 2 ' a ([0, +00) x R m ). 

In this case the solutions correspond to critical points of the functional 

$(u) = i / |W| 2 dx- / g{\x\,U)dx 




in H Q (f2) and the linearized operator (13.1 p coincides with the second 
derivative of $. 

Thus standard variational methods apply which often give solutions 
of finite (linearized) Morse index, as, for example, in the case when 
the Mountain Pass Theorem can be used or when one considers the so 
called "least energy" nodal solutions. So, if the hypotheses of Theorem 
11.21 are satisfied, our symmetry results can be applied (see also [7]). 

A second type of interesting systems of two equations are the so 
called "Hamiltonian type" systems (see [5] and the references therein). 
More precisely we consider the system 

/i(|a;|,Wi,U2) in Q 
(4.1) { -Au 2 = f 2 (\x\,Ui,u 2 ) in Q 

on dQ 

with 
(4.2) 

dH dH 

fl(\x\,U!,U2) = {\x\,Ui,U 2 ) , f 2 {\x\,U 1 ,U 2 ) = ^— (\x\,Ui,U2) 
OU2 OUi 

for some scalar function H G C 2,a ([0,+oo) x R 2 ). These systems can 
be studied by considering the associated functional 

(4.3) J(U) = I(u±, u 2 ) = — [ Vu\-Vu 2 dx— f H(\x\, u±, u 2 ) dx 

2 Jq Jn 

either in H (fl) or in other suitable Sobolev spaces (see [H]). 

It is easy to see that the linearized operator defined in (13. ip does 
not correspond to the second derivative of the functional J, which 
is strongly indefinite. Nevertheless solutions of (14. ip can have finite 
linearized Morse index as shown in [7j. 

In particular we consider the system 

= \u 2 \ p ~ 1 u 2 in Q 

(4.4) { -Au 2 =\u l \ q - 1 u 1 mQ 

= on an 

where 1 < p, q < ]Pj§. Then we start from the case p = q and the 
solution u\ = u 2 = z, where z is a scalar solution of the equation 




(4.5) 



-Az = \z\ v ~ x z in Q 
= on dQ 
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Let us observe that if p = q and z has Morse index equal to the 
integer fi{z), then n(z) is also the Morse index of the solution U = 
(uijUi) = (z,z) of the system (14. 4p . Indeed the linearized equation 
at z for the equation (14. 5 p and the linearized system at (z, z) for the 
system (I4.4p are respectively 



-A(j>- plzl'f- 1 ^ =0inQ 

= o on an 



(4.6) 
and 

{-A0! - p|z| p "V2 = in VI in ft 
-A02 - pH P ~Vi = in VI in ft 
4>i = 4>2 = on <9ft 

This implies that the eigenvalues of these two operators are the same, 
since if (J) is an eigenfunction for (14. 6 p corresponding to the eigenvalue 
Afc then taking <f>i = 2 = 4> we obtain an eigenfunction (<pi, for (14. 7p 
corresponding to the same eigenvalue, while if ((pi, 2 ) is an eigenfunc- 
tion for (14. 7p corresponding to the eigenvalue then </> — (j>x + 2 is 
an eigenfunction for (14. 6 p corresponding to the same eigenvalue. 

So if we start from a nondegenerate solution of (14. 5 p with a fixed 
exponent p G (1, -j^rf), using the Implicit Function Theorem, we find a 
branch of solutions of (I4.4p corresponding to (possibly different) expo- 
nents p, q close to p. For example if we start with a least energy nodal 
solution z in the ball of equation (14. 5 p with the exponent p, knowing 
that its Morse index is two we get a branch of Morse index two solutions 
for p, q close to p. 

Note that, as proved in [2], the least energy nodal solution of (14. 5p is 
not radial but foliated Schwarz symmetric. So it is obviously degener- 
ate, but working in the space of axially symmetric functions we could 
remove the degeneracy and apply the continuation method described 
above, if there are no other degeneracies. 

Thus, starting from an exponent p for which the least energy nodal 
solution of (14. 5 p is not degenerate, we can construct solutions U = 
(ui,U2) of (14.41) in correspondence of exponents p, q close to p, with 
Morse index two. Then Theorem 11.21 applies if p, q > 2 and, in partic- 
ular, we get that the coupling condition (I1.5P holds, which in this case 
can be written as 

(4.8) p|w 2 r _1 = gKI 9 " 1 in n 

Note that more generally, by Theorem II .3[ the equality (14. 8 p must hold 
for every solution U = (u\, u 2 ) of (14. 4p with Morse index n{U) < N—l, 
giving so a sharp condition to be satisfied by the components of a 
solution of this type. 

Let us remark that our results apply also when the nonlinearity de- 
pends on |x| (in any way). Arguing as before it is not difficult to 
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construct systems having solutions with low Morse index, in particular 
with Morse index one or two. 

An example could be the " Henon system" 



(4.9) 



— Aui = \x\ a 
-Au 2 = \x\P\ 
u±, u 2 > 
u\ = u=0 



p-i 

9-1 



U 2 
Ui 



in Q 
in Q 
in Q 

on d£l 



with a, (3 > 0, p, q > 2. 
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